Learning Deep Policies for Physics-Based Manipulation in Clutter

Uncertainty in modeling real world physics makestransferring traditional open-loop motion planning techniquesfrom simulation to the real world particularly challenging.Available closed-loop policy learning approaches, for physics-based manipulation tasks, typically either focus on single objectmanipulation, or rely on imitation learning, which inherentlyconstrains task generalization and performance to the availabledemonstrations. In this work, we propose an approach to learna policy for physics-based manipulation in clutter, which enablesthe robot to react to the uncertain dynamics of the real world.We start with presenting an imitation learning technique whichcompiles demonstrations from a sampling-based planner intoan action-value function encoded as a deep neural network. Wethen use the learned action-value function to guide a look-aheadplanner, giving us a control policy. Lastly, we propose to refinethe deep action-value function through reinforcement learning,taking advantage of the look-ahead planner. We evaluate ourapproach in a physics-enabled simulation environment withartificially injected uncertainty, as well as in a real world taskof manipulation in clutter.